disagg: Disagg write filecache for compaction tasks (release-nextgen-20251011) by JaySon-Huang · Pull Request #10940 · pingcap/tiflash

JaySon-Huang · 2026-06-29T15:36:45Z

manually cherry-pick of #10937

What problem does this PR solve?

Issue Number: close #xxx

Problem Summary:

What is changed and how it works?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

Initialize FileCache and register remote cache paths for disaggregated storage nodes, and add dt_enable_write_filecache (default false) as the runtime gate for upcoming write-side local staging.

Propagate an optional DMFileBlockInputStreamBuilder hook through getPlacedStream and enable it from prepareMerge, prepareMergeDelta, and prepareSplitPhysical when dt_enable_write_filecache is set.

Introduce collectMetaV2MergedFilesForLocalRead to map read columns to deduplicated physical .merged S3 objects, with unit tests for noop, column filtering, and remote MetaV2 dedup behavior.

Download deduplicated .merged S3 objects before building DMFileReader when write FileCache local read is enabled, pin FileSegmentPtr for reader lifetime, and fall back to direct read on per-object failures.

Replace ProfileEvents with tiflash_storage_write_filecache_staging counters and staged-bytes counter, and update local staging unit tests accordingly.

Cover prepareMergeDelta, prepareMerge, and prepareSplitPhysical staging paths in SegmentTestS3, and fix FileCache test teardown to re-init S3FileCachePool so multiple gtests can run in one process.

Signed-off-by: JaySon-Huang <tshent@qq.com>

…logging Instrument prepareMergeDelta/Merge/SplitPhysical and remote DMFile upload with tiflash_storage_subtask_* metrics. Track remote upload duration in createNewStable and log stable column count in Segment::info().

Return prepare/remote upload durations from prepareMerge and prepareMergeDelta, and print them in segmentMerge/MergeDelta finish logs.

Track prepare and S3 upload durations in SplitInfo and print them in segmentSplit finish logs. Move getPlacedStream above the gtest visibility split in Segment.h.

Signed-off-by: JaySon-Huang <tshent@qq.com>

Pass DownloadType into downloadImpl and print it in Download success logs.

Collect large standalone `.dat` subfiles in addition to `.merged` blobs when preparing write-side FileCache local reads, with unit test coverage.

Signed-off-by: JaySon-Huang <tshent@qq.com>

ti-chi-bot · 2026-06-29T15:36:49Z

This cherry pick PR is for a release branch and has not yet been approved by triage owners.
Adding the do-not-merge/cherry-pick-not-approved label.

To merge this cherry pick:

It must be LGTMed and approved by the reviewers firstly.
For pull requests to TiDB-x branches, it must have no failed tests.
AFTER it has lgtm and approved labels, please wait for the cherry-pick merging approval from triage owners.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ti-chi-bot · 2026-06-29T15:36:53Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign solotzg, yudongusa for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS
dbms/src/Interpreters/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coderabbitai · 2026-06-29T15:36:54Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (3)

release-8.5
release-7.5
release-8.1

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 8ef98088-7b96-4216-88ce-cdc7383cca0a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Signed-off-by: JaySon-Huang <tshent@qq.com>

…te+mergeDeltaAll (pingcap#10898) close pingcap#10897\n\nSigned-off-by: JaySon-Huang <tshent@qq.com>

JaySon-Huang · 2026-06-30T05:35:03Z

close as not need

JaySon-Huang added 17 commits June 29, 2026 23:34

storage: enable FileCache on disagg storage nodes

75bd8fd

Initialize FileCache and register remote cache paths for disaggregated storage nodes, and add dt_enable_write_filecache (default false) as the runtime gate for upcoming write-side local staging.

storage: wire write FileCache builder hook for background reads

b97fcf8

Propagate an optional DMFileBlockInputStreamBuilder hook through getPlacedStream and enable it from prepareMerge, prepareMergeDelta, and prepareSplitPhysical when dt_enable_write_filecache is set.

storage: add MetaV2 merged file collector for write FileCache staging

1b5c65d

Introduce collectMetaV2MergedFilesForLocalRead to map read columns to deduplicated physical .merged S3 objects, with unit tests for noop, column filtering, and remote MetaV2 dedup behavior.

storage: stage MetaV2 merged files via FileCache in DMFileReader

85bf22d

Download deduplicated .merged S3 objects before building DMFileReader when write FileCache local read is enabled, pin FileSegmentPtr for reader lifetime, and fall back to direct read on per-object failures.

storage: use TiFlash metrics for write FileCache staging

092469a

Replace ProfileEvents with tiflash_storage_write_filecache_staging counters and staged-bytes counter, and update local staging unit tests accordingly.

storage: add segment integration tests for write FileCache staging

d04e529

Cover prepareMergeDelta, prepareMerge, and prepareSplitPhysical staging paths in SegmentTestS3, and fix FileCache test teardown to re-init S3FileCachePool so multiple gtests can run in one process.

Storages: Add dmf_disk_bytes to logging

7e2e5de

Signed-off-by: JaySon-Huang <tshent@qq.com>

storage: log prepare and remote upload timing in merge finish

e0da308

Return prepare/remote upload durations from prepareMerge and prepareMergeDelta, and print them in segmentMerge/MergeDelta finish logs.

storage: log prepare and remote upload timing in split finish

865773b

Track prepare and S3 upload durations in SplitInfo and print them in segmentSplit finish logs. Move getPlacedStream above the gtest visibility split in Segment.h.

Format codes

2e5dc31

Signed-off-by: JaySon-Huang <tshent@qq.com>

disable flaky fap test

8800b34

Signed-off-by: JaySon-Huang <tshent@qq.com>

storage: tag FileCache downloadImpl with foreground/background type

1814fd9

Pass DownloadType into downloadImpl and print it in Download success logs.

storage: stage standalone MetaV2 column files for write FileCache

65d944e

Collect large standalone `.dat` subfiles in addition to `.merged` blobs when preparing write-side FileCache local reads, with unit test coverage.

storage: log elapsed time for slow split ingest after 10s

6cc0912

storage: add ingestFiles duration metric and elapsed logging

41a1981

Format codes

6b08e65

Signed-off-by: JaySon-Huang <tshent@qq.com>

ti-chi-bot Bot added do-not-merge/cherry-pick-not-approved release-note-none Denotes a PR that doesn't merit a release note. labels Jun 29, 2026

ti-chi-bot Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jun 29, 2026

JaySon-Huang added 2 commits June 30, 2026 09:26

tests: disable flaky fap test

4f49df1

Signed-off-by: JaySon-Huang <tshent@qq.com>

test: fix flaky DisaggReadSnapshot — skip background tasks during wri…

063328c

…te+mergeDeltaAll (pingcap#10898) close pingcap#10897\n\nSigned-off-by: JaySon-Huang <tshent@qq.com>

JaySon-Huang closed this Jun 30, 2026

JaySon-Huang deleted the ng_2510_disagg-write-filecache branch June 30, 2026 05:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

disagg: Disagg write filecache for compaction tasks (release-nextgen-20251011)#10940

disagg: Disagg write filecache for compaction tasks (release-nextgen-20251011)#10940
JaySon-Huang wants to merge 19 commits into
pingcap:release-nextgen-20251011from
JaySon-Huang:ng_2510_disagg-write-filecache

JaySon-Huang commented Jun 29, 2026

Uh oh!

ti-chi-bot Bot commented Jun 29, 2026

Uh oh!

ti-chi-bot Bot commented Jun 29, 2026

Uh oh!

coderabbitai Bot commented Jun 29, 2026 •

edited

Loading

Review skipped

Uh oh!

JaySon-Huang commented Jun 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

JaySon-Huang commented Jun 29, 2026

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

Uh oh!

ti-chi-bot Bot commented Jun 29, 2026

Uh oh!

ti-chi-bot Bot commented Jun 29, 2026

Uh oh!

coderabbitai Bot commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

JaySon-Huang commented Jun 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coderabbitai Bot commented Jun 29, 2026 •

edited

Loading